Orthogonal Forward Regression based on Directly Maximizing Model Generalization Capability

The paper introduces a construction algorithm for sparse kernel modelling using the leave-one-out test score also known as the PRESS (Predicted REsidual Sums of Squares) statistic. An efficient subset model selection procedure is developed in the orthogonal forward regression framework by incrementally maximizing the model generalization capability to construct sparse models with good generalization properties. The proposed algorithm achieves a fully automated model construction without resort to any other validation data set for costly model evaluation. Index Terms — orthogonal forward regression, structure identification, cross validation, generalization.